Economic Evaluations of CAR-T Cell Therapies for Hematologic and Solid Malignancies: A Systematic Review.

OBJECTIVES: This study aims to systematically review evidence on the cost-effectiveness of chimeric antigen receptor (CAR)-T therapies for patients with cancer. METHODS: Electronic databases were searched in October 2022 and updated in September 2023. Systematic reviews, health technology assessments and economic evaluations that compared costs and effects of CAR-T therapy in cancer patients were included. Two reviewers independently screened studies, extracted data, synthesized results, and critically appraised studies using the Philips checklist. Cost data were presented in 2022 US Dollars RESULTS: Our search yielded 1,809 records, 47 of which were included. The majority of included studies were cost-utility analysis, published between 2018 and 2023, and conducted in the United States. Tisagenlecleucel, axicabtagene ciloleucel, idecabtagene vicleucel, ciltacabtagene autoleucel, lisocabtagene maraleucel, brexucabtagene autoleucel, and relmacabtagene autoleucel were compared to various standard-of-care chemotherapies. The incremental cost-effective ratio (ICER) for CAR-T therapies ranged from $9,424 to $4,124,105 per QALY in adults and from $20,784 to $243,177 per QALY in pediatric patients. ICERs were found to improve over longer time horizons or when an earlier cure point was assumed. Most studies failed to meet the Philips checklist due to a lack of head-to-head comparisons and uncertainty surrounding CAR-T costs and curative effects. CONCLUSIONS: CAR-T therapies were more expensive and generated more QALYs than comparators, but their cost-effectiveness were uncertain and dependent on patient population, cancer type, and model assumptions. This highlights the need for more nuanced economic evaluations and continued research to better understand the value of CAR-T therapies in diverse patient populations.

Implementation of an audit and feedback module targeting low-value clinical practices in a provincial trauma quality assurance program: a cost-effectiveness study.

BACKGROUND: Audit and Feedback (A&F) interventions based on quality indicators have been shown to lead to significant improvements in compliance with evidence-based care including de-adoption of low-value practices (LVPs). Our primary aim was to evaluate the cost-effectiveness of adding a hypothetical A&F module targeting LVPs for trauma admissions to an existing quality assurance intervention targeting high-value care and risk-adjusted outcomes. A secondary aim was to assess how certain A&F characteristics might influence its cost-effectiveness. METHODS: We conducted a cost-effectiveness analysis using a probabilistic static decision analytic model in the Québec trauma care continuum. We considered the Québec Ministry of Health perspective. Our economic evaluation compared a hypothetical scenario in which the A&F module targeting LVPs is implemented in a Canadian provincial trauma quality assurance program to a status quo scenario in which the A&F module is not implemented. In scenarios analyses we assessed the impact of A&F characteristics on its cost-effectiveness. Results are presented in terms of incremental costs per LVP avoided. RESULTS: Results suggest that the implementation of A&F module (Cost = $1,480,850; Number of LVPs = 6,005) is associated with higher costs and higher effectiveness compared to status quo (Cost = $1,124,661; Number of LVPs = 8,228). The A&F module would cost $160 per LVP avoided compared to status quo. The A&F module becomes more cost-effective with the addition of facilitation visits; more frequent evaluation; and when only high-volume trauma centers are considered. CONCLUSION: A&F module targeting LVPs is associated with higher costs and higher effectiveness than status quo and has the potential to be cost-effective if the decision-makers' willingness-to-pay is at least $160 per LVP avoided. This likely represents an underestimate of true ICER due to underestimated costs or missed opportunity costs. Results suggest that virtual facilitation visits, frequent evaluation, and implementing the module in high-volume centers can improve cost-effectiveness.

The Ottawa Statement implementation guidance document for cluster randomized trials in the hemodialysis setting.

Research teams are increasingly interested in using cluster randomized trial (CRT) designs to generate practice-guiding evidence for in-center maintenance hemodialysis. However, CRTs raise complex ethical issues. The Ottawa Statement on the Ethical Design and Conduct of Cluster Randomized Trials, published in 2012, provides 15 recommendations to address ethical issues arising within 7 domains: justifying the CRT design, research ethics committee review, identifying research participants, obtaining informed consent, gatekeepers, assessing benefits and harms, and protecting vulnerable participants. But applying the Ottawa Statement recommendations to CRTs in the hemodialysis setting is complicated by the unique features of the setting and population. Here, with the help of content experts and patient partners, we co-developed this implementation guidance document to provide research teams, research ethics committees, and other stakeholders with detailed guidance on how to apply the Ottawa Statement recommendations to CRTs in the hemodialysis setting, the result of a 4-year research project. Thus, our work demonstrates how the voices of patients, caregivers, and all stakeholders may be included in the development of research ethics guidance.

Describing the content of trial recruitment interventions using the TIDieR reporting checklist: a systematic methodology review.

BACKGROUND: Recruiting participants to clinical trials is an ongoing challenge, and relatively little is known about what recruitment strategies lead to better recruitment. Recruitment interventions can be considered complex interventions, often involving multiple components, targeting a variety of groups, and tailoring to different groups. We used the Template for Intervention Description and Replication (TIDieR) reporting checklist (which comprises 12 items recommended for reporting complex interventions) to guide the assessment of how recruitment interventions are described. We aimed to (1) examine to what extent we could identify information about each TIDieR item within recruitment intervention studies, and (2) observe additional detail for each item to describe useful variation among these studies. METHODS: We identified randomized, nested recruitment intervention studies providing recruitment or willingness to participate rates from two sources: a Cochrane review of trials evaluating strategies to improve recruitment to randomized trials, and the Online Resource for Research in Clinical triAls database. First, we assessed to what extent authors reported information about each TIDieR item. Second, we developed descriptive categorical variables for 7 TIDieR items and extracting relevant quotes for the other 5 items. RESULTS: We assessed 122 recruitment intervention studies. We were able to extract information relevant to most TIDieR items (e.g., brief rationale, materials, procedure) with the exception of a few items that were only rarely reported (e.g., tailoring, modifications, planned/actual fidelity). The descriptive variables provided a useful overview of study characteristics, with most studies using various forms of informational interventions (55%) delivered at a single time point (90%), often by a member of the research team (59%) in a clinical care setting (41%). CONCLUSIONS: Our TIDieR-based variables provide a useful description of the core elements of complex trial recruitment interventions. Recruitment intervention studies report core elements of complex interventions variably; some process elements (e.g., mode of delivery, location) are almost always described, while others (e.g., duration, fidelity) are reported infrequently, with little indication of a reason for their absence. Future research should explore whether these TIDieR-based variables can form the basis of an approach to better reporting of elements of successful recruitment interventions.

One size doesn't fit all: methodological reflections in conducting community-based behavioural science research to tailor COVID-19 vaccination initiatives for public health priority populations.

BACKGROUND: Promoting the uptake of vaccination for infectious diseases such as COVID-19 remains a global challenge, necessitating collaborative efforts between public health units (PHUs) and communities. Applied behavioural science can play a crucial role in supporting PHUs' response by providing insights into human behaviour and informing tailored strategies to enhance vaccination uptake. Community engagement can help broaden the reach of behavioural science research by involving a more diverse range of populations and ensuring that strategies better represent the needs of specific communities. We developed and applied an approach to conducting community-based behavioural science research with ethnically and socioeconomically diverse populations to guide PHUs in tailoring their strategies to promote COVID-19 vaccination. This paper presents the community engagement methodology and the lessons learned in applying the methodology. METHODS: The community engagement methodology was developed based on integrated knowledge translation (iKT) and community-based participatory research (CBPR) principles. The study involved collaboration with PHUs and local communities in Ontario, Canada to identify priority groups for COVID-19 vaccination, understand factors influencing vaccine uptake and co-design strategies tailored to each community to promote vaccination. Community engagement was conducted across three large urban regions with individuals from Eastern European communities, African, Black, and Caribbean communities and low socioeconomic neighbourhoods. RESULTS: We developed and applied a seven-step methodology for conducting community-based behavioural science research: (1) aligning goals with system-level partners; (2) engaging with PHUs to understand priorities; (3) understanding community strengths and dynamics; (4) building relationships with each community; (5) establishing partnerships (community advisory groups); (6) involving community members in the research process; and (7) feeding back and interpreting research findings. Research partnerships were successfully established with members of prioritized communities, enabling recruitment of participants for theory-informed behavioural science interviews, interpretation of findings, and co-design of targeted recommendations for each PHU to improve COVID-19 vaccination uptake. Lessons learned include the importance of cultural sensitivity and awareness of sociopolitical context in tailoring community engagement, being agile to address the diverse and evolving priorities of PHUs, and building trust to achieve effective community engagement. CONCLUSION: Effective community engagement in behavioural science research can lead to more inclusive and representative research. The community engagement approach developed and applied in this study acknowledges the diversity of communities, recognizes the central role of PHUs, and can help in addressing complex public health challenges.

Evidence for clinician underprescription of and patient non-adherence to guideline-recommended cardiovascular medications among adults with peripheral artery disease: protocol for a systematic review and meta-analysis.

INTRODUCTION: International guidelines recommend that adults with peripheral artery disease (PAD) be prescribed antiplatelet, statin and antihypertensive medications. However, it is unclear how often people with PAD are underprescribed these drugs, which characteristics predict clinician underprescription of and patient non-adherence to guideline-recommended cardiovascular medications, and whether underprescription and non-adherence are associated with adverse health and health system outcomes. METHODS AND ANALYSIS: We will search MEDLINE, EMBASE and Evidence-Based Medicine Reviews from 2006 onwards. Two investigators will independently review abstracts and full-text studies. We will include studies that enrolled adults and reported the incidence and/or prevalence of clinician underprescription of or patient non-adherence to guideline-recommended cardiovascular medications among people with PAD; adjusted risk factors for underprescription of/non-adherence to these medications; and adjusted associations between underprescription/non-adherence to these medications and outcomes. Outcomes will include mortality, major adverse cardiac and limb events (including revascularisation procedures and amputations), other reported morbidities, healthcare resource use and costs. Two investigators will independently extract data and evaluate study risk of bias. We will calculate summary estimates of the incidence and prevalence of clinician underprescription/patient non-adherence across studies. We will also conduct subgroup meta-analyses and meta-regression to determine if estimates vary by country, characteristics of the patients and treating clinicians, population-based versus non-population-based design, and study risks of bias. Finally, we will calculate pooled adjusted risk factors for underprescription/non-adherence and adjusted associations between underprescription/non-adherence and outcomes. We will use Grading of Recommendations, Assessment, Development and Evaluation to determine estimate certainty. ETHICS AND DISSEMINATION: Ethics approval is not required as we are studying published data. This systematic review will synthesise existing evidence regarding clinician underprescription of and patient non-adherence to guideline-recommended cardiovascular medications in adults with PAD. Results will be used to identify evidence-care gaps and inform where interventions may be required to improve clinician prescribing and patient adherence to prescribed medications. PROSPERO REGISTRATION NUMBER: CRD42022362801.

Protocol for the ONLOOP trial: pragmatic randomized trial evaluating a province-wide system of personalized reminders for evidence-based surveillance tests in adult survivors of childhood cancer in Ontario.

BACKGROUND: Childhood cancer treatment while often curative, leads to elevated risks of morbidity and mortality. Survivors require lifelong periodic surveillance for late effects of treatment, yet adherence to guideline-recommended tests is suboptimal. We created ONLOOP to provide adult survivors of childhood cancer with detailed health information, including summaries of their childhood cancer treatment and recommended surveillance tests for early detection of cardiomyopathy, breast cancer, and/or colorectal cancer, with personalized reminders over time. METHODS: This is an individually randomized, registry-based pragmatic trial with an embedded process and economic evaluation to understand ONLOOP's impact and whether it can be readily implemented at scale. All adult survivors of childhood cancer in Ontario overdue for guideline-recommended tests will be randomly assigned to one of two arms: (1) intervention or (2) delayed intervention. A letter of information and invitation will detail the ONLOOP program. Those who sign up will receive a personalized toolkit and a screening reminder 6 months later. With the participants' consent, ONLOOP will also send their primary care clinician a letter detailing the recommended tests and a reminder 6 months later. The primary outcome will be the proportion of survivors who complete one or more of the guideline-recommended cardiac, breast, or colon surveillance tests during the 12 months after randomization. Data will be obtained from administrative databases. The intent-to-treat principle will be followed. Based on our analyses of administrative data, we anticipate allocating at least 862 individuals to each trial arm, providing 90% power to detect an absolute increase of 6% in targeted surveillance tests completed. We will interview childhood cancer survivors and family physicians in an embedded process evaluation to examine why and how ONLOOP achieved success or failed. A cost-effectiveness evaluation will be performed. DISCUSSION: The results of this study will determine if ONLOOP is effective at helping adult survivors of childhood cancer complete their recommended surveillance tests. This study will also inform ongoing provincial programs for this high-risk population. TRIAL REGISTRATION: ClinicalTrials.gov NCT05832138.

Developing a set of emergency department performance measures to evaluate delirium care quality for older adults: a modified e-Delphi study.

BACKGROUND: Older adults are at high risk of developing delirium in the emergency department (ED); however, it is under-recognized in routine clinical care. Lack of detection and treatment is associated with poor outcomes, such as mortality. Performance measures (PMs) are needed to identify variations in quality care to help guide improvement strategies. The purpose of this study is to gain consensus on a set of quality statements and PMs that can be used to evaluate delirium care quality for older ED patients. METHODS: A 3-round modified e-Delphi study was conducted with ED clinical experts. In each round, participants rated quality statements according to the concepts of importance and actionability, then their associated PMs according to the concept of necessity (1-9 Likert scales), with the ability to comment on each. Consensus and stability were evaluated using a priori criteria using descriptive statistics. Qualitative data was examined to identify themes within and across quality statements and PMs, which went through a participant validation exercise in the final round. RESULTS: Twenty-two experts participated, 95.5% were from west or central Canada. From 10 quality statements and 24 PMs, consensus was achieved for six quality statements and 22 PMs. Qualitative data supported justification for including three quality statements and one PM that achieved consensus slightly below a priori criteria. Three overarching themes emerged from the qualitative data related to quality statement actionability. Nine quality statements, nine structure PMs, and 14 process PMs are included in the final set, addressing four areas of delirium care: screening, diagnosis, risk reduction and management. CONCLUSION: Results provide a set of quality statements and PMs that are important, actionable, and necessary to a diverse group of clinical experts. To our knowledge, this is the first known study to develop a de novo set of guideline-based quality statements and PMs to evaluate the quality of delirium care older adults receive in the ED setting.

The Quality in Acute Stroke Care (QASC) global scale-up using a cascading facilitation framework: a qualitative process evaluation.

BACKGROUND: Variation in hospital stroke care is problematic. The Quality in Acute Stroke (QASC) Australia trial demonstrated reductions in death and disability through supported implementation of nurse-led, evidence-based protocols to manage fever, hyperglycaemia (sugar) and swallowing (FeSS Protocols) following stroke. Subsequently, a pre-test/post-test study was conducted in acute stroke wards in 64 hospitals in 17 European countries to evaluate upscale of the FeSS Protocols. Implementation across countries was underpinned by a cascading facilitation framework of multi-stakeholder support involving academic partners and a not-for-profit health organisation, the Angels Initiative (the industry partner), that operates to promote evidence-based treatments in stroke centres. .We report here an a priori qualitative process evaluation undertaken to identify factors that influenced international implementation of the FeSS Protocols using a cascading facilitation framework. METHODS: The sampling frame for interviews was: (1) Executives/Steering Committee members, consisting of academics, the Angels Initiative and senior project team, (2) Angel Team leaders (managers of Angel Consultants), (3) Angel Consultants (responsible for assisting facilitation of FeSS Protocols into multiple hospitals) and (4) Country Co-ordinators (senior stroke nurses with country and hospital-level responsibilities for facilitating the introduction of the FeSS Protocols). A semi-structured interview elicited participant views on the factorsthat influenced engagement of stakeholders with the project and preparation for and implementation of the FeSS Protocol upscale. Interviews were recorded, transcribed verbatim and analysed inductively within NVivo. RESULTS: Individual (n = 13) and three group interviews (3 participants in each group) were undertaken. Three main themes with sub-themes were identified that represented key factors influencing upscale: (1) readiness for change (sub-themes: negotiating expectations; intervention feasible and acceptable; shared goal of evidence-based stroke management); (2) roles and relationships (sub-themes: defining and establishing roles; harnessing nurse champions) and (3) managing multiple changes (sub-themes: accommodating and responding to variation; more than clinical change; multi-layered communication framework). CONCLUSION: A cascading facilitation model involving a partnership between evidence producers (academic partners), knowledge brokers (industry partner, Angels Initiative) and evidence adopters (stroke clinicians) overcame multiple challenges involved in international evidence translation. Capacity to manage, negotiate and adapt to multi-level changes and strategic engagement of different stakeholders supported adoption of nurse-initiated stroke protocols within Europe. This model has promise for other large-scale evidence translation programs.

Evidence Synthesis for Complex Interventions Using Meta-Regression Models.

A goal of evidence synthesis for trials of complex interventions is to inform the design or implementation of novel versions of complex interventions by predicting expected outcomes with each intervention version. Conventional aggregate data meta-analyses of studies comparing complex interventions have limited ability to provide such information. We argue that evidence synthesis for trials of complex interventions should forgo aspirations of estimating causal effects and instead model the response surface of study results to 1) summarize the available evidence and 2) predict the average outcomes of future studies or in new settings. We illustrate this modeling approach using data from a systematic review of diabetes quality improvement (QI) interventions involving at least 1 of 12 QI strategy components. We specify a series of meta-regression models to assess the association of specific components with the posttreatment outcome mean and compare the results to conventional meta-analysis approaches. Compared with conventional approaches, modeling the response surface of study results can better reflect the associations between intervention components and study characteristics with the posttreatment outcome mean. Modeling study results using a response surface approach offers a useful and feasible goal for evidence synthesis of complex interventions that rely on aggregate data.

Living evidence and adaptive policy: perfect partners?

BACKGROUND: While there has been widespread global acceptance of the importance of evidence-informed policy, many opportunities to inform health policy with research are missed, often because of a mismatch between when and where reliable evidence is needed, and when and where it is available. 'Living evidence' is an approach where systematic evidence syntheses (e.g. living reviews, living guidelines, living policy briefs, etc.) are continually updated to incorporate new relevant evidence as it becomes available. Living evidence approaches have the potential to overcome a major barrier to evidence-informed policy, making up-to-date systematic summaries of policy-relevant research available at any time that policy-makers need them. These approaches are likely to be particularly beneficial given increasing calls for policy that is responsive, and rapidly adaptive to changes in the policy context. We describe the opportunities presented by living evidence for evidence-informed policy-making and highlight areas for further exploration. DISCUSSION: There are several elements of living approaches to evidence synthesis that might support increased and improved use of evidence to inform policy. Reviews are explicitly prioritised to be 'living' by partnerships between policy-makers and researchers based on relevance to decision-making, as well as uncertainty of existing evidence, and likelihood that new evidence will arise. The ongoing nature of the work means evidence synthesis teams can be dynamic and engage with policy-makers in a variety of ways over time; and synthesis topics, questions and methods can be adapted as policy interests or contextual factors shift. Policy-makers can sign-up to be notified when relevant new evidence is found, and can be confident that living syntheses are up-to-date and contain all research whenever they access them. The always up-to-date nature of living evidence syntheses means producers can rapidly demonstrate availability of relevant, reliable evidence when it is needed, addressing a frequently cited barrier to evidence-informed policymaking. CONCLUSIONS: While there are challenges to be overcome, living evidence provides opportunities to enable policy-makers to access up-to-date evidence whenever they need it and also enable researchers to respond to the issues of the day with up-to-date research; and update policy-makers on changes in the evidence base as they arise. It also provides an opportunity to build flexible partnerships between researchers and policy-makers to ensure that evidence syntheses reflect the changing needs of policy-makers.

Identifying behaviour change techniques in 287 randomized controlled trials of audit and feedback interventions targeting practice change among healthcare professionals.

BACKGROUND: Audit and feedback (A&F) is among the most widely used implementation strategies, providing healthcare professionals with summaries of their practice performance to prompt behaviour change and optimize care. Wide variability in effectiveness of A&F has spurred efforts to explore why some A&F interventions are more effective than others. Unpacking the variability of the content of A&F interventions in terms of their component behaviours change techniques (BCTs) may help advance our understanding of how A&F works best. This study aimed to systematically specify BCTs in A&F interventions targeting healthcare professional practice change. METHODS: We conducted a directed content analysis of intervention descriptions in 287 randomized trials included in an ongoing Cochrane systematic review update of A&F interventions (searched up to June 2020). Three trained researchers identified and categorized BCTs in all trial arms (treatment & control/comparator) using the 93-item BCT Taxonomy version 1. The original BCT definitions and examples in the taxonomy were adapted to include A&F-specific decision rules and examples. Two additional BCTs ('Education (unspecified)' and 'Feedback (unspecified)') were added, such that 95 BCTs were considered for coding. RESULTS: In total, 47/95 BCTs (49%) were identified across 360 treatment arms at least once (median = 5.0, IQR = 2.3, range = 129 per arm). The most common BCTs were 'Feedback on behaviour' (present 89% of the time; e.g. feedback on drug prescribing), 'Instruction on how to perform the behaviour' (71%; e.g. issuing a clinical guideline), 'Social comparison' (52%; e.g. feedback on performance of peers), 'Credible source' (41%; e.g. endorsements from respected professional body), and 'Education (unspecified)' (31%; e.g. giving a lecture to staff). A total of 130/287 (45%) control/comparator arms contained at least one BCT (median = 2.0, IQR = 3.0, range = 0-15 per arm), of which the most common were identical to those identified in treatment arms. CONCLUSIONS: A&F interventions to improve healthcare professional practice include a moderate range of BCTs, focusing predominantly on providing behavioural feedback, sharing guidelines, peer comparison data, education, and leveraging credible sources. We encourage the use of our A&F-specific list of BCTs to improve knowledge of what is being delivered in A&F interventions. Our study provides a basis for exploring which BCTs are associated with intervention effectiveness. TRIAL REGISTRATIONS: N/A.

Protocol for a living evidence synthesis on variants of concern and COVID-19 vaccine effectiveness.

BACKGROUND: It is evident that COVID-19 will remain a public health concern in the coming years, largely driven by variants of concern (VOC). It is critical to continuously monitor vaccine effectiveness as new variants emerge and new vaccines and/or boosters are developed. Systematic surveillance of the scientific evidence base is necessary to inform public health action and identify key uncertainties. Evidence syntheses may also be used to populate models to fill in research gaps and help to prepare for future public health crises. This protocol outlines the rationale and methods for a living evidence synthesis of the effectiveness of COVID-19 vaccines in reducing the morbidity and mortality associated with, and transmission of, VOC of SARS-CoV-2. METHODS: Living evidence syntheses of vaccine effectiveness will be carried out over one year for (1) a range of potential outcomes in the index individual associated with VOC (pathogenesis); and (2) transmission of VOC. The literature search will be conducted up to May 2023. Observational and database-linkage primary studies will be included, as well as RCTs. Information sources include electronic databases (MEDLINE; Embase; Cochrane, L*OVE; the CNKI and Wangfang platforms), pre-print servers (medRxiv, BiorXiv), and online repositories of grey literature. Title and abstract and full-text screening will be performed by two reviewers using a liberal accelerated method. Data extraction and risk of bias assessment will be completed by one reviewer with verification of the assessment by a second reviewer. Results from included studies will be pooled via random effects meta-analysis when appropriate, or otherwise summarized narratively. DISCUSSION: Evidence generated from our living evidence synthesis will be used to inform policy making, modelling, and prioritization of future research on the effectiveness of COVID-19 vaccines against VOC.

Assessing the feasibility, acceptability, and fidelity of a tele-retinopathy-based intervention to encourage greater attendance to diabetic retinopathy screening in immigrants living with diabetes from China and African-Caribbean countries in Ottawa, Canada: a protocol.

BACKGROUND: Diabetic retinopathy is a leading cause of preventable blindness in Canada. Clinical guidelines recommend annual diabetic retinopathy screening for people living with diabetes to reduce the risk and progression of vision loss. However, many Canadians with diabetes do not attend screening. Screening rates are even lower in immigrants to Canada including people from China, Africa, and the Caribbean, and these groups are also at higher risk of developing diabetes complications. We aim to assess the feasibility, acceptability, and fidelity of a co-developed, linguistically and culturally tailored tele-retinopathy screening intervention for Mandarin-speaking immigrants from China and French-speaking immigrants from African-Caribbean countries living with diabetes in Ottawa, Canada, and identify how many from each population group attend screening during the pilot period. METHODS: We will work with our health system and patient partners to conduct a 6-month feasibility pilot of a tele-retinopathy screening intervention in a Community Health Centre in Ottawa. We anticipate recruiting 50-150 patients and 5-10 health care providers involved in delivering the intervention for the pilot. Acceptability will be assessed via a Theoretical Framework of Acceptability-informed survey with patients and health care providers. To assess feasibility, we will use a Theoretical Domains Framework-informed interview guide and to assess fidelity, and we will use a survey informed by the National Institutes of Health framework from the perspective of health care providers. We will also collect patient demographics (i.e., age, gender, ethnicity, health insurance status, and immigration information), screening outcomes (i.e., patients with retinopathy identified, patients requiring specialist care), patient costs, and other intervention-related variables such as preferred language. Survey data will be descriptively analyzed and qualitative data will undergo content analysis. DISCUSSION: This feasibility pilot study will capture how many people living with diabetes from each group attend the diabetic retinopathy screening, costs, and implementation processes for the tele-retinopathy screening intervention. The study will indicate the practicability and suitability of the intervention in increasing screening attendance in the target population groups. The study results will inform a patient-randomized trial, provide evidence to conduct an economic evaluation of the intervention, and optimize the community-based intervention.

What is the role of randomised trials in implementation science?

BACKGROUND: There is a consistent demand for implementation science to inform global efforts to close the gap between evidence and practice. Key evaluation questions for any given implementation strategy concern the assessment and understanding of effects. Randomised trials are generally accepted as offering the most trustworthy design for establishing effectiveness but may be underused in implementation science. MAIN BODY: There is a continuing debate about the primacy of the place of randomised trials in evaluating implementation strategies, especially given the evolution of more rigorous quasi-experimental designs. Further critiques of trials for implementation science highlight that they cannot provide 'real world' evidence, address urgent and important questions, explain complex interventions nor understand contextual influences. We respond to these critiques of trials and highlight opportunities to enhance their timeliness and relevance through innovative designs, embedding within large-scale improvement programmes and harnessing routine data. Our suggestions for optimising the conditions for randomised trials of implementation strategies include strengthening partnerships with policy-makers and clinical leaders to realise the long-term value of rigorous evaluation and accelerating ethical approvals and decluttering governance procedures for lower risk studies. CONCLUSION: Policy-makers and researchers should avoid prematurely discarding trial designs when evaluating implementation strategies and work to enhance the conditions for their conduct.

"I think there has to be a mutual respect for there to be value": Evaluating patient engagement in a national clinical trial on de-implementation of low value care.

BACKGROUND: The evaluation of patient engagement in research is understudied and under-reported, making it difficult to know what engagement strategies work best and when. We provide the results of an evaluation of patient engagement in a large Canadian research program focused on the de-implementation of low-value care. We aimed to evaluate the experience and impact of patient engagement in the study. METHODS: An online cross-sectional survey was administered using Microsoft Forms to (1) researchers and study staff and (2) patient partners. The survey was developed following iterative reviews by the project's patient partnership council and evaluation committee. Survey content areas included opinions on patient engagement to date, including challenges to engagement and suggestions for improvement. Patient partners also evaluated the partnership council. Descriptive statistics including counts and percentages described Likert scale survey items, while open comments were analyzed using descriptive content analysis. RESULTS: The survey response rate was 46% (17/37). There were positive attitudes about the value of patient engagement in this project. There was also a high degree of willingness to be involved with patient engagement in future projects, whether as a patient partner or as a researcher including patients on the research team. Most patient partners felt their contributions to the project were valued by researchers and study research staff. Open comments revealed that a co-design approach and full inclusion on the research team were integral to demonstrating the value of patient partner input. Areas for improvement included more frequent and ongoing communication among all team members, as well as earlier training about patient engagement, particularly addressing role expectations and role clarity. CONCLUSIONS: Our data revealed that despite some challenges, team members recognized the value of patient engagement in research and agreed project decisions had been impacted by patient partner input. Ongoing communication was highlighted as an area for improvement, as well as earlier training and ongoing support for all team members, but particularly researchers and study staff. In response to evaluation data, the team has reinstated a quarterly newsletter and plans to use specific patient engagement planning templates across study sites for all project activities. These tools should help make expectations clear for all team members and contribute to a positive patient engagement experience. Findings can inform patient engagement planning and evaluation for other health research projects.

Evaluating patient engagement in research is often not done or not reported, making it hard to know what engagement strategies work best and when. Here, we provide the results of an evaluation of patient engagement in a Canadian Institutes for Health Research (CIHR) Strategy for Patient Oriented Research (SPOR) Innovative Clinical Trial Multi-Year Grant. The project focuses on strategies to reduce two low-value care practices (pre-operative testing in low-risk day surgery and imaging for low back pain). An online survey was sent to project researchers, study staff and patient partners to get their opinions on the patient engagement in the project. Generally, there were positive attitudes about the value of patient engagement in the project. Both patient partners and research study staff were very willing to be involved with patient engagement in future projects. Most patient partners felt their contributions to the project were valued by researchers and study research staff. An important part of showing the value of patient partners was working together to design the project and making sure that the patient partners were considered full members of the research team. Areas for improvement included better communication among all team members and earlier training about patient engagement with a focus on patient roles and expectations. The results from this survey will be used to improve the patient engagement in this project but will also help patient engagement planning and evaluation for other health research projects.